Skip to content

Hi Goodtimes - Compute qualified-event mask from each pointing's matched gain config - #3625

Merged
tmplummer merged 2 commits into
IMAP-Science-Operations-Center:devfrom
tmplummer:3618-hi-goodtimes---compute-qualified-event-mask-from-each-pointings-matched-gain-config
Nov 6, 2026
Merged

tmplummer merged 2 commits into
IMAP-Science-Operations-Center:devfrom
tmplummer:3618-hi-goodtimes---compute-qualified-event-mask-from-each-pointings-matched-gain-config

Conversation

@tmplummer

@tmplummer tmplummer commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Change Summary

Overview

Closes #3618. Goodtimes now qualifies each pointing's events using only the cal-prod rows of that pointing's matched gain config. Surrounding pointings that match no gain config get no qualified events and are excluded from Statistical Filter 1. The overflow packet filter likewise uses only the current pointing's matched gain config.

File changes

  • hi_goodtimes.py: new _add_qualified_event_masks() (per-pointing gain-config selection) and _get_gain_match_values() helper; Filter 1 is passed only matched pointings plus the current one. mark_overflow_packets uses the matched gain config's coincidence types.
  • test_hi_goodtimes.py: new tests for per-gain-config masks and Filter 1 pointing selection; existing _apply_goodtimes_filters tests updated.

Testing

  • Added a two-gain-config test with non-overlapping TOF windows, plus unmatched-pointing and Filter 1 selection tests. Added a two-gain-config overflow test. All 439 Hi tests pass.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Because the change affects Hi science-quality qualification and Statistical Filter 1 across gain configurations, it requires instrument-team/runtime validation that automated review cannot provide for the full suite and representative products.

No open issues 🎉

What changed in this PR

Updates Hi Goodtimes to qualify events using only the gain configuration matched to each pointing, preventing unrelated calibration windows from affecting Statistical Filter 1.

Changes:

  • Added per-pointing gain-configuration matching and qualified-event masks.
  • Excluded unmatched surrounding pointings from Statistical Filter 1 while retaining the current pointing.
  • Added tests for multiple gain configurations, unmatched pointings, invalid events, and filter-1 dataset selection.
File Description
imap_processing/​tests/​hi/​test_hi_goodtimes.py Adds coverage for gain-specific qualification and unmatched-pointing behavior.
imap_processing/​hi/​hi_goodtimes.py Implements gain-specific event masks and matched-pointing selection for Statistical Filter 1.

Review effort: Lite

@leowerneck

Copy link
Copy Markdown
Contributor

I started to review it. I will need more time to digest everything, but in the meantime I'll share one of the findings from my local agent:

Before, `_apply_goodtimes_filters` only runs when all 9 datasets are present,
so Filter 1 always got 9 datasets. Now, if 6 or more of the 8 surrounding pointings
are unmatched, Filter 1 raises, failing the whole job. Whether this is a realistic
outcome or not needs to be checked, but it could happen to matched pointings
near the switchover.

@tmplummer

Copy link
Copy Markdown
Contributor Author

I started to review it. I will need more time to digest everything, but in the meantime I'll share one of the findings from my local agent:

Before, `_apply_goodtimes_filters` only runs when all 9 datasets are present,
so Filter 1 always got 9 datasets. Now, if 6 or more of the 8 surrounding pointings
are unmatched, Filter 1 raises, failing the whole job. Whether this is a realistic
outcome or not needs to be checked, but it could happen to matched pointings
near the switchover.

There should almost never be an unmatched pointing. I believe there is one to date that occurred on the pointing where the gain configuration of Hi was updated during the uplink contact. In general though, a unmatched pointing will be rare and only caused by live commanding or an anomaly. Raising in this situation is the appropriate thing to do.

@leowerneck leowerneck left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

I have only one non-blocking comment/question: the new code refactored the calculation of hv_deltas as the _get_gain_match_values() function. I like this, but from the diff it seems like we went from performing the operation once in mark_unmatched_gain_config() to three times [added work is now in _add_qualified_event_masks() and mark_overflow_packets()]. I just wanted to bring this up to double check that this small amount additional work is indeed intended, or if we would prefer to just pass the single result around.

@tmplummer

Copy link
Copy Markdown
Contributor Author

I have only one non-blocking comment/question: the new code refactored the calculation of hv_deltas as the _get_gain_match_values() function. I like this, but from the diff it seems like we went from performing the operation once in mark_unmatched_gain_config() to three times [added work is now in _add_qualified_event_masks() and mark_overflow_packets()]. I just wanted to bring this up to double check that this small amount additional work is indeed intended, or if we would prefer to just pass the single result around.

Yeah, I see what you mean. Really only two of these are duplicated work because the _add_qualified_event_masks() function gets applied to each of the 9 pointings of l1b_de product input. Although it would be possible to store these values, the amount of duplicated work is tiny (retrieving 4 attribute values and filtering a small dataframe) and the tradeoff for keeping the function signatures simpler is, IMO, better in this case.

@tmplummer
tmplummer merged commit c4bf1d6 into IMAP-Science-Operations-Center:dev Nov 6, 2026
22 checks passed
@tmplummer
tmplummer deleted the 3618-hi-goodtimes---compute-qualified-event-mask-from-each-pointings-matched-gain-config branch November 6, 2026 18:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Hi Goodtimes - Compute qualified-event mask from each pointing's matched gain config

3 participants